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INTRODUCTION 



Angular catalogs of galaxy positions have proven a very use- 
ful tool with which to study the statistical properties of large 
scale density fluctuations, as they provide large volume cov- 
erage, so that local density fluctuations in the nearby galaxy 
distribution are averaged out. The isotropy and large scale 
homogeneity of the universe allows the recovery of the under- 
lying spatial statistics. However, in the derivation of three- 
dimensional (3D) properties from angular data there are 
large uncertainties which arise from the selection function 
and the evolution of the galaxy population. These uncer- 
tainties must be considered carefully for an accurate inter- 
pretation of the underlying clustering properties. 

We concentrate on the APM Galaxy Survey (Maddox 
et al. 1990a-c) of angular positions and, in particular, on the 
bj = 17 — 20 subsample which has over 1.3 x 10 6 galaxies and 
a mean depth V ~ 400 h' 1 Mpc. Maddox et al. (1990a) have 
proposed a redshift dependent luminosity function model 
as a rough approximation to the selection function used to 
estimate the 3D clustering in the APM. Here we use new 
observational constraints on the local luminosity function 



We analyze the uncertainties in the amplitudes of the spatial correlation functions 
estimated from angular correlations in a sample from the APM Galaxy Survey, with 
bj = 17 — 20. We model the uncertainties in the selection function and in the evolution 
of clustering. In particular we estimate a^ PM , the rms galaxy number fluctuations in 
spheres of radius at 8 Mpc, from the measured angular variance in the APM. The 
uncertainty in a^ PM has three main contributions: 8% from sampling and selection 
function uncertainties, 7% from the uncertainty in the evolution of clustering and 3% 
from the uncertainty in the value of Slo- Including all these contributions, we find <Tg PM 
is in the range 0.78 — 1.08. If the galaxy clustering in the APM evolves as expected from 
gravitational clustering of matter fluctuations, then a^ PM = 0.95±0.07 (1.00±0.08) for 
Slo — 1 (^o — 0), close to the values for nearby optical samples. On the other hand, if we 
assume that clustering evolution is fixed in comoving coordinates a^ PM = 0.83 ± 0.05 
(0.87 ± 0.06), closer to the results for nearby IRAS samples. The final uncertainty in 

the range of values for the hierarchical amplitudes Sj = £j/£2 i s typically twice 
the estimated sampling errors, with the highest values for the case of less clustering 
evolution. We compare our estimates with other results and discuss the implications for 
models of structure formation. 
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(Loveday et al. 1992) to study the range of possible selec- 
tion functions and their effect on the estimated 3D cluster- 
ing properties, also extending the analysis to higher order 
correlations. 

We will pay special attention to the normalization of 
clustering amplitudes at small scales, where power-laws 
are a good approximation. As the APM covers the largest 
volume sampled to date, well over 1.5 x 10 8 (h~ 1 Mpc) 3 , an 
accurate determination of these quantities should provide 
important constraints on models of structure formation. 

The projection effects and the recovery of 3-D clustering 
are presented in § 2. In § 3 we discuss limits on the selection 
function of our sample by modelling the luminosity function, 
while in § 4 we consider models for the clustering evolution. 
We apply these models to the problem of inversion from the 
APM data in § 5 and present a final discussion in § 6. 



2 PROJECTION EFFECTS 

We now present a simple method for recovering the 3-D 
variance, £ 2 (i?), and higher order moments, £ j{R), from the 
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2-D correlations, uij(9). We will use the same notation and 
definitions as in Gaztanaga (1994, hereafter G94). 

2.1 Scale- invariant model 

Consider first the following scale-invariant model for the cor- 
relations: 
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(e.g. Fry & Peebles 1978, Fry 1984b). Here the product 
of the two-point functions, ^(rij), is over ,1 — 1 indepen- 
dent pairs of relative separations and the sum, consisting 
of ,1 ~ 2 terms, is over equivalent reassignments of labels 
i, j = 1,2,3 J. The amplitudes Q j are just numbers that 
can be generalized to Qj, a where a denotes different topolo- 
gies in the graphs connecting the labels. Thus, the hierarchy 
in equation (2) is composed of "tree" graphs (connected with 
no cycles) of I vertices and ,1 — 1 edges. 

Observations indicate that the above hierarchy holds at 
least for lower values of ,1 , at small scales (Groth & Peebles 
1977; Fry & Peebles 1978) or up to J = 10 when averaged 
over all scales (Szapudi, Szalay & Boshan 1992; Meiksin, 
Szapudi & Szalay 1992, Szapudi et al. 1995). This same hier- 
archy has also been obtained from theoretical considerations 
(Davis & Peebles 1977, Fry 1984a, Hamilton 1988, Balian & 
Schaeffer 1989a, 1989b). In perturbation theory (PT), similar 
hierarchical forms to equation (2) have been found (Peebles 
1980, Fry 1984b, Goroff et al. 1986, Bernardeau 1994). In 
this case, the hierarchical parameters Qj are not constant, 
but depend on the configuration arrangements and, in par- 
ticular, on the scale. Analyses of N-body simulations show 
that these analytical results are accurate on large scales for 
smoothed correlations { j ( Juszkiewicz, Bouchet & Colombi 
1993, Bernardeau 1994, Juszkiewicz et al. 1995, Lokas et 
al. 1995, Gaztanaga & Baugh 1995, Baugh, Gaztanaga & 
Efstathiou 1995). 

The above model produces the following volume aver- 
aged correlations in a spherical cell (top-hat window smooth- 
ing) of radius R: 
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tj(R) = SjfaiR)]'- 1 , 1>2 



The smoothed amplitudes, as and Sj are related to the 
multi-point amplitudes, ro and Qj, by: 
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where Bj ~ 1 are given in G94 (see also Boschan, Szapudi 
& Szalay 1994). 

For small angles this scale invariant model produces the 
following angular averaged correlations: 

TJ 2 (9) = A 9 1_1 (6) 



ZJj(0) = sjp 2 (6)] J -\ 1>2 

The angular amplitudes, A and sj are related to the multi- 
point values, ro and Qj, by: 
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where Cj ~ 1 are given in G94 (see also Boschan, Szapudi 
& Szalay 1994) and T 7 is a geometrical factor that comes 
from the area average in T02 '■ 
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Note that the rj in equation (7) are dimensionless and 
are not directly related to ro (which has units of ft -1 Mpc). 
The values oi Ik in equation (7), can be expressed as: 



F(x) x 2 dx j," x (3 - 1 '> (k - 1 '> (1 + 2 )(3+--0(i-*) (9) 



where x is the comoving coordinate. These integrals de- 
pend on the selection function ij)(x), the curvature correction 
F 2 (x) = [1 — (fio — l)(-Ho£/c) 2 ] an d the evolution of cluster- 
ing, parametrized by e (which will be discussed in the next 
sections). 

Thus for a given scale-invariant model with slope 7, it is 
possible to use the above expressions to relate the estimated 
angular amplitudes A and sj, in equation (7), to the un- 
derlying three dimensional amplitudes, i.e. as and Sj, from 
equations 4 and 5. 



2.2 Quasi-scale-invariant model 

Consider now a distribution that is not exactly scale- 
invariant but has correlations that can be parametrized 
as a scale-invariant distribution as in equation (2) with ro, 
7 and Q j being a slowly varying function of scale. We call 
this a quasi-scale-invariant model. 

For a a quasi-scale-invariant model it should be possi- 
ble to apply a local inversion at each scale. In principle the 
correlations on all scales R contribute to the correlations on 
angular scale 9, but because the sample has a finite depth, 
V ', there is a characteristic scale R ~ V9. In our analysis we 
relate angular scales 9 to 3-D scales using R = T>9 , where D 
is the estimated distance which corresponds to the mean red- 
shift of the sample (see also Peebles 1980). Although there 
is some ambiguity as to what the best definition of D should 
be, in the scale-invariant regime, we find that the estimated 
amplitudes of are insensitive to changes in our chosen 
value of D . 

Thus at a given scale 9 with local slope 7, we use the 
above expressions to relate the estimated local angular am- 
plitudes A and sj to the underlying three dimensional val- 
ues, i.e. as and Sj at scale R = T>9 . This model was used in 
G94 to recover the 3D correlations in the APM Survey. 
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2.3 Tests on N-body simulations 



We have tested the quasi-scale invariant method on sev- 
eral simulations with different shapes and ampfitudes for the 
variance f 2 . We have tried both galaxy and cluster simula- 
tions, generated by Gaztanaga & Baugh (1995) and Croft & 
Efstathiou (1994a, b). We first estimate { j using the counts 
in cells method for the whole simulation box (as in Baugh, 
Gaztanaga & Efstathiou 1995). Next we transform the sim- 
ulation into an observational catalogue with a given selec- 
tion function. ^From this mock catalogue we estimate To j 
and use the above method to recover £j. The comparison 
(Baugh & Gaztanaga, in preparation) shows excellent agree- 
ment within the errors even at large scales, where there is a 
significant break from the power-law model. 

The simulations we use have values of Sj which show a 
small variation with scale, e.g. S3 ~ R a , with a ~ 0.1 (Fig. 
3 in Gaztanaga & Baugh 1995), and strictly speaking nei- 
ther the scale-invariant nor the quasi-scale-invariant models 
should be used, as Sj should be constants. Nevertheless, we 
still find reasonable agreement from the inversion when we 
compare local values of Sj. These results and further details 
are presented elsewhere. 
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2.4 Test on the APM galaxy data 

We now apply this method to the APM galaxies and com- 
pare it with a the results of a previous estimate by Baugh & 
Efstathiou (1993). We first estimate the shape of the vari- 
ance, £ 2 , in the APM Galaxy Survey by integrating the 
three-dimensional P(k) measured by Baugh & Efstathiou 
(1994), i.e. 

?2 = f 2 dkk 2 P(k)W 2 (kR), (10) 

where W(kR) is the Fourier transform of the spherical win- 
dow with radius R 

W(kR) = j^^[sm(kR) - kRcos(kR)]. (11) 

The errors correspond to 2a scatter in the angular two-point 
correlation function (Maddox et al. 1990a). The results are 
shown as filled symbols in Figure 1. 

The second estimate is based on the method presented 
above for quasi-scale-invariant models. We use the angular 
variance measured in the APM (i.e. Figure 5). Our final 
errors include the sampling errors in T02 and the uncertainty 
in the local slope. In both cases we use the selection function 
of Baugh & Efstathiou (1993) and the same model for the 
evolution of clustering (e = 0, see below). Open squares 
with error bars in Figure 1 shows this new estimate of f 2 
compared to that from P(k). Both results agree perfectly 
within the errors. 



3 THE SELECTION FUNCTION 

The selection function ip( x ) i s the normalized probability 
that a galaxy at coordinate x is included in the catalogue. 
This probability is proportional to the estimated number of 
galaxies at this coordinate: 



Figure 1. Comparison of two different estimates of £ 2 from the 
angular APM sample of galaxies. Filled squares are obtained 
by integrating the power spectrum P(k) of Baugh & Efstathiou 
(1993). Open squares correspond to a direct inversion of the an- 
gular variance ~ 2 (6) . 



/•92O) 

Mx) = r dq<t>(q) (12) 

J q 1 ( x) 

where ip* is adjusted so that the probability integrates to 
unity over the sample. (f>(q) is the luminosity function and 
qi(x) and 52(2;) are the scaled luminosities corresponding to 
the lower and upper limits in the range of apparent mag- 
nitudes used to build the galaxy sample or catalog under 
study. In our case these are bj = 17 and bj = 20 respec- 
tively. The luminosity function (f>(q) in equation (12) should 
include all photometric contamination that has not been cor- 
rected for in the magnitudes in our sample, i.e. (f>(q) should 
not have been corrected for magnitude errors, k-corrections 
or extinction. This is done in practice by introducing an 
observer magnitude system M' so that the absolute magni- 
tude, M , of a galaxy at redshift z is given by M' = M — kz. 
We use k = 3 as a rough approximation but it should not 
affect our results much because we allow all our parame- 
ters have a redshift dependence. We also use the standard 
Schechter form for the luminosity function: 

<j,(q) = fq a e -l , q = lQ-i(M'-M') (13) 

An alternative to this approach of estimating ip( x ) i s 
to have a direct measurement of the redshift distribution 
N(z) dz, i.e. the number of galaxies at a given redshift z (as 
in Baugh & Efstathiou 1993). However, a redshift catalog of 
APM galaxies in the range b j = 17 — 20 is not yet available, 
and instead we will have to extrapolate ip( x ) from luminosity 
function estimations. 
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3.1 Constraints on the local luminosity 
function 

In the analysis of a bright sample of the APM galaxies 
(with bj < 17.15), the Stromlo-APM Redshift Survey, Love- 
day et al. (1992) found that the values of the parameters 
in the luminosity function (uncorrected for magnitude er- 
rors) are: a = -1.11 ± 0.15, M * = -19.73 ± 0.13 and 
<f>o = 1.12 ± 0.12 x W~ 2 h 3 Mpc" 3 , where the errors in <f>* 
are basically due to the uncertainties in a and M*. The me- 
dian redshift of the Stromlo-APM sample is only z ~ 0.05 
but the volume is large enough to avoid clustering effects 
or large scale density fluctuations. Similar results have re- 
cently been obtained by Vettolani et al. (1995) over a sample 
from the ESO key-project. If we apply these parameters for 
4>(q) to the bj = 17 — 20 APM sample we find that it pre- 
dicts a total number of ~ 740, 000 galaxies. This is almost 
a factor of two smaller than the actual measured number, 
~ 1, 300, 000 galaxies. This seems to support the conclusion 
of Maddox et al. (1990b) that the rapid increase in number 
counts in the magnitude range bj = 16 — 19 can only be ex- 
plained by significant evolution of the galaxy population at 
redshifts z ~ 0.1. This could be a consequence of luminosity 
evolution or a decrease in the number density (see Koo and 
Kron 1992 and Colles 1994, Glazebrook et al. 1995a, 1995b 
for a recent discussion), but the observations could also be 
affected by important selection effects (e.g. Salzer 1994, Mc- 
Gaugh 1994). Koo, Gronwall & Bruzual (1993) and Gron- 
wall & Koo (1995) have proposed that traditional luminosity 
evolution could explain the observations if we assume a par- 
ticular (non Schechter) local luminosity function, with more 
galaxies (than in Loveday et al. extrapolation) at the faint 
end, Mb > —16.5, where there are no direct observations. 

Whatever the reason, we will consider a rapid change in 
the effective luminosity function to parametrize the redshift 
distribution N(z) in our magnitude range. As pointed out 
above, this approach might not be the most general possi- 
bility, but it seems that it can reproduce the observed N(z) 
distributions in the cases where data is available. 



3.2 Constraints on evolution with redshift 

We have parametrized the redshift evolution by making (f>* = 
(f>*(z), M* = M*(z) and a = a(z) function of the redshift z. 
Because the redshifts are small z < 1 we use linear functions: 



M* 
a 



Mo +M* z , 

a + «1 z , 

4>o ( i + 4>* z ) 



(14) 



so that the dimensionless parameters M* , ct\ and <j>\ mea- 
sure the redshift dependence. In order to set some con- 
straints on the parameter space we use the following results: 

1) The Stromlo-APM values of the luminosity function 
at low redshifts from Loveday et al. (1992) are taken as 
corresponding to z ~ 0. Therefore, we take: 



a = -1.11 ±0.15 
Mo = -19.73 ±0.13 
4>*o = (1.12 ±0.12) x 10~ 2 h 3 Mpc~ 




Figure 2. Allowed regions for different amounts of evolution as 
a function of M* , a\ and two values of </>*: top </>* = and bot- 
tom </>* = 5. In all cases the values of Mq and ao are allowed to 
vary within their uncertainties. The region within the two contin- 
uous lines corresponds to the constraint from the total number of 
galaxies, N17—20 = 1.30 ± 0.13 X 10 6 . The two long-dashed lines 
enclose the region with mean redshift ~20—21.5 = 0.22 ± 0.01, 
while the two short-dashed lines enclose the region with mean 
redshift ~ 2 l-22.5 = 0.31 ± 0.02 



2) The total number of galaxies in the bj = 17 — 20 
APM sample is: 

A17-20 = 1.30 ± 0.08 x 10 6 . (16) 

This number has been obtained directly from the APM 
maps and the error is based on a merging correction 
uncertainty of 5%, i.e. Maddox et al. (1990a). 

3) The mean redshift at bj = 20 — 21.5 found by Broad- 
hurst et al. (1988) is: 



220-21.5 



0.22 ± 0.01. 



4) The mean redshift at bj 
et al. (1990) is: 

~21-22.5 = 0.31 ± 0.02. 



(17) 

21-22.5 found by Colless 



(18) 



(15) 



This list is somewhat arbitrary and does not pretend to 
include all observations on galaxy counts or galaxy redshifts. 
One could also consider other constraints, such as the slope 
of the galaxy counts at faint magnitudes bj ~ 21, but these 
are usually subject to larger uncertainties. 

In Figure 2 we show the allowed regions in M*-a± space 
for <j>1 = and (f>* = 5. Assuming the no-evolution param- 
eters Mo, 4>o and ao in equation (15) the allowed regions 
for each equation (16), equation (17) and equation (18) cor- 
respond to the space between the continuous, long-dashed 
and short-dashed lines respectively in Figure 2. These re- 
gions correspond to 2 standard deviations (uncertainties are 



On the interpretation of clustering from the angular APM Galaxy Survey 5 



o„=i 



n„=o 



01 = 5 





01 = 2 



<P] = 

J L 




Figure 3. x 2 = 1.32(75%), 2. 71(90%), 3.84(95%) contours for dif- 
ferent luminosity function parameters M* and a\ , and three val- 
ues of </>*: </>* = 0, </>* = 2 and </>* = 5. In all cases Qq = 1. 



Figure 4. x 2 contours for different luminosity function parame- 
ters as in as Figure 3, but with Qq = 



added in quadrature). The no evolution model (which cor- 
responds to M* = 0, a\ = and <j>* = 0) is outside the 
allowed region because it produces too few galaxies, but it 
is compatible with the mean redshift values. Models that 
allow only evolution in a (which correspond to M* = and 
4>* = 0) are not able to produce enough extra galaxies with- 
out a considerable change in the mean redshift. Models that 
have evolution only in 0* (which corresponds to M* = 
and a\ = 0) require <j>* ~ 5 which involves a change of a 
factor of 2 in <j>* from 2 = to z = 0.2. 

We furthermore construct a \ 2 test to find contours of 
the values that best fit the constraints. The \ 2 test has three 
contributions, corresponding to the differences between the 
predicted and observed values of N g , "20-21.5 and "21-22.5- 

The x 2 = 1-32(75%), 2.71(90%), 3.84(95%) contours are 
shown in Figure 3 for fio = 1. The best fit, at a 75% 
level of confidence (\ 2 J; 1-32), corresponds to cf>* ~ 0, 
«i £ [-4., -3.5] and M* £ [-2.05, -1.85]. At a 90% level of 
confidence the allowed regions include other values of <j>*. For 
(f>* = 5 we have «i £ [-1.4,-0.3] and M* £ [-0.12,0.17]. 
The contours for intermediate values of <j>* change smoothly 
as M* increases from <j>* = to <j>* = 5, following the trend 
indicated by Figure 3. 

The results in Figure 3 correspond to the mean values 
of Mq , «o and <j>*. The uncertainties in these values can be 
translated roughly into further uncertainties in M* and a\ 
as AM* ~ AMo /z and A«i ~ Aa /z. For z ~ 0.15, in our 
sample, we have AM* ~ 0.9 and Aai ~ 1, comparable with 
the values shown by the contours. 

The above results depend slighly on the value of fio • As 
we change fio from 1 to the contours move slowly to the 
right, with mean values a\ that follow ai(Qo) ~ ai(0) — fio- 
There is also a smaller trend upwards, which is correlated 
with changes in <j>*, so that mean values of M* follow 
Mf(Q ) ~ Mj*(0) - Q o 0i*/2O. The results for Q = are 
shown in Figure 4. 



We find x 2 ^ 15.6 (Q = 0) or X 2 ^ 28 (Q = 1) for 
the luminosity function proposed by Maddox et al. (1990a), 



i.e. in our notation MX 



-19.8, a 



-1, M* 



-2 



and «i = —2 normalized using <j>* from equation (15). This 
model has been used by Maddox et al. and by G94 as a rough 
approximation to the selection function when estimating the 
three-dimensional clustering in the APM. The reason for the 
large \ 2 ls that this model predicts larger mean redshifts 
than those observed: "20-21.5 — 0.26 for fio = or fio = 1 
and "21-22.5 — 0.34(Q = 1) or "21-22.5 — 0.33(Q = 0). 
With such a large \ 2 it is clear that this model does not fit 
well the constraints we have chosen and it is important to 
see how different the resulting clustering predictions are. 

Baugh & Efstathiou (1993) have proposed a functional 
form for the redshift distribution N(z) which provides an ac- 
ceptable match to the deep redshift histograms (Broadhurst 
et al. 1988 and Colless et al. 1990) whilst simultaneously fit- 
ting the Stromlo/APM redshift distribution (Loveday et al. 
1992). By construction one can also match the total number 
in the bj = 17 - 20 APM sample. We find that this N (z) 
distribution gives "20-21.5 — 0.23 and "21-22.5 — 0.29, and 
this contributes to giving an acceptable \ 2 — 2.23. Nev- 
ertheless, when using this functional form for N(z) we do 
not explore the uncertainty in the selection function, which 
is what we need to do in order to study its effect on the 
projected clustering. 



4 EVOLUTION OF CLUSTERING 

As usual, we parametrize the evolution of clustering using: 

{ 2 (r, 2 ) = (l + 2 )- (£+3) 6(r), (19) 

where ^(t) corresponds to z = 0. For the higher order cor- 
relations we assume that the evolution follows from the hier- 
archical model equation (2). That is, that Qj and Sj do not 
evolve much with z. This is in good agreement with N-body 
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simulations (see Baugh, Gaztanaga & Efstathiou 1995 and 
Gaztanaga & Baugh 1995). 

When the intrinsic clustering properties do not evolve in 
proper coordinates (stable clustering) the excess probability 
of finding a galaxy at separation r from a given galaxy is a 
constant: 

n(z) £,2{r,z) = n &(r) = const, (20) 

where no corresponds to n(z) at z = 0. Because of the Uni- 
versal expansion n(z) = rao(l + z) 3 , so that: 

t2(r,z) = (l + z)- 3 t 2 (r), (21) 

corresponding to e = in equation (19). Thus ^(t, z) is 
smaller by a factor of ~ 1.7 at z ~ 0.02. 

However, clustering evolves under gravity. When {2 < 1, 
i.e. at large scales r > To, the clustering is near the linear 
regime ^(x) = a 2 ^2( x )j where ^(x) are the initial condi- 
tions. Assuming that over some range in scale the initial con- 
ditions are scale invariant: ^(x) ~ £ -7 , we have e = 7 — 1 
in equation (19). /.From observational galaxy catalogs (see 
below) it is found that 7 ~ 2 for scales r > To increasing to 
larger values at larger scales, so that e > 1 or larger. Thus in 
the linear regime e > 1 and ^(t, z) evolves more than in the 
stable clustering regime. In the small scale regime, where 
£2^1, N-body simulations typically show more clustering 
evolution than in the linear regime (i.e. Baugh, Gaztanaga 
& Efstathiou 1995, see also Figures 11-12), and therefore 
e > 7 — 1. Note nevertheless that these arguments apply 
to the matter distribution, while we are interested in the 
galaxy distribution. 

If the clustering pattern is fixed in comoving coordi- 
nates, then e = 7 — 3 ~ — 1.3, with less evolution than in the 
stable clustering regime. This might describe some models 
in which galaxies are identified with high density peaks. As 
one would expect peaks to move less than mass particles it 
results in less evolution for the clustering of galaxies. 

Although there are several observations of clustering of 
faint galaxies they do not seem to set a definitive constraint 
on e. We will consider the range e ~ —1.3 to e ~ 1.3 below, 
in order to include all the above possibilities. 



5 CLUSTERING IN THE APM SURVEY 

We use here the clustering results from the angular APM 
Galaxy Survey (Maddox et al. 1990a), in particular for the 
bj = 17 — 20 sample which has over 1.3 x 10 6 galaxies. The 
area-averaged angular J-point correlation functions Zoj(9) 
were estimated for J = 2 — 9 by G94. We have recalculated 
here these correlations Zoj(9) using more cell sizes so that 
we have better scale resolution (each cell is 50%, instead 100 
%, larger than the previous one). 

5.1 The Variance: ^ 2 

5.1.1 The 2-point amplitude: To and as 

Figure 5 shows the estimated values of Z02. This figure is 
similar to Figure 1 of G94, but with more cell sizes and with 
errors from the variance in 4 random subsets (and not 4 
zones). The possible effects of any artificial gradients in the 
APM caused by plate matching errors could introduce the 
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Figure 5. Variance of angular counts, W2(0), for the bj = 17 — 20 
APM sample. Errors are estimated from the variance in 4 random 
subsets. The continuous line models the effect of a possible artifi- 
cial gradient in the APM Survey caused by plate matching errors 
(derived using the model of Maddox et al. 1990c). The dashed line 
shows the result of substracting this correction from our mean es- 
timate. 



following contribution to the two-point angular correlation 
function: 

Aw 2 (9 12 ) < 2 x 10" 3 /[1 + (#i2/6°) 2 ]. (22) 

This function varies from 2 x 10 -3 at small angular 
scales to 1 x 10 -3 on the plate scale 812 — 6° in the model 
for large scale gradients described by Maddox et al. (1990c, 
see also Baugh & Efstathiou 1993). The corresponding an- 
gular variance AZ02 in circular cells of radius 9 is shown as 
a continuous line in Figure 5. The possible net effect on the 
measured T02 is shown as a dashed line. The resulting angular 
variance lies within the estimated sampling errors. Other 
uncertainties in the construction of the APM Galaxy Survey 
(such as other magnitude errors or non-uniform star-galaxy 
separation) are likely to give even smaller errors (Maddox 
et al. , 1990c). 

A least-squares fit to a power-law To(9) = A 9~^ for 
small scales, 9 = 0.09 — 0.9 in degrees, gives fi ~ 0.7 ± 
0.2 and A = (3.82 ± 0.12) x 10" 2 , where we have applied 
a merging correction of 5 %, (see Maddox et al. (1990a)). 
[Note that this correction has not been applied to the values 
shown in Figure 5.] This value of A is used in equation (7) 
to find ro. Given a model for clustering evolution, i.e. e in 
equation (19), the value of ro depends on each parameter 
«i, M* and <j>\ in (f>(L,z), ( equation (14)). In Figure 6 we 
show these values of ro as contours in the a\-M* plane for 



* Note that at large scales, where this effect is larger, our final 
errors in £ 2 are dominated by the uncertainty in the value of the 
local slope 7. These final errors are much larger than the artificial 
gradient correction, see Figures 7-8. 
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Figure 6. Contours of values of r for the APM bj = 17 - 20 
galaxy sample using different luminosity functions, i.e. values of 
a.\ and M* , and </>* = 0. The thick continuous line marks the 
contour where r$ = 5 h~ 1 Mpc, above this line r$ < 5 /i — 1 Mpc. 
Other boundaries are marked at intervals of 0.2 h — 1 Mpc (long- 
dashed lines correspond to r$ < 5 h~ 1 Mpc, short-dashed lines 
to tq > 5 h~ 1 Mpc ). Each panel shows a different model for 
the evolution of clustering labelled by t and Q. Overlaid closed 
contours show the allowed regions according to the number counts 
and mean redshift, i.e. Figure 3. 

Table 1. Estimated values of <7g in the APM for different models 
for the evolution of clustering and different values of Qo • 

ai PM Q = 1 fio = 

e = +1.3 0.95 ± 0.07 1.00 ± 0.08 
e = 0.0 0.89 ± 0.06 0.93 ± 0.07 
e = -1.3 0.83 ± 0.05 0.87 ± 0.06 

e = -3.0 0.75 ± 0.05 0.79 ± 0.06 



different e and two vafues of fio • The aiiowed contours from 
Figure 3 are piaced on top of the ro vafues. 

Figure 6 corresponds to the case (f>* = 0, but overaif, 
for any given e the finai aiiowed values of ro are roughly 
independent of (f>*. 

It is also interesting to express the 2-point amplitude in 
terms of the variance <rf in spheres of radius 8 ft -1 Mpc, i.e. 
equation (4). In Table 1 we show the best fit values of <r&. 




i i i mm i i i i iii t~ 

10 100 
R (Mpc hr 1 ) 

Figure 7. Comparison of different estimates of £ 2 from direct 
inversion of the angular variance ~ 2 (#) for the two extreme sit- 
uations: i) Q = 1 and e = —1.3 (open squares) ii) Q = and 
e = +1.3 (filled squares). In both cases </>* = and M* = —2. 
The inset is an enlarged view of a portion of the same graph 
around R ~ 8 Mpc. 



For each value of e and fio we find the most likely value of 
ro and its uncertainty for all possible values of M* , a\ and 
(f>*. The resulting uncertainties are then added in quadrature 
with the sampling errors in A (~ 2%) to estimate ag from 
equation (4) -we use the slope at 1°, 7 = 1.70 ± 0.02, which 
corresponds to R ~ 8 ft -1 Mpc. Thus the resulting errors in 
Table 1 include sampling errors, uncertainties in the slope, 
and uncertainties in the selection function. This adds up to 
an ~ 8% error in <jg, for any given value of e and fio- There 
is 7% variation of <jg with e (for e between -1.3 and 1.3) and 
a 3% variation with fio- 

In Table 1 we also show the results for e = — 3, which 
corresponds to the case of no clustering evolution and no 
cosmic expansion. 



5.1.2 The shape of f 2 

We next use the inversion presented in §2 to estimate f 2 at 
different scales. 

Figure 7 shows the extreme case e = +1.3 and Q = 
as compared to e = —1.3 and Q = 1. In both cases, we use 
the luminosity function model with (f>* = and M* = —2, 
with a\ = —4 for Q = 1 and a\ = —3 for Q = 0, suggested 
by figures 3-4. Results for other models for the luminosity 
function or evolution of clustering give intermediate values 
(see Figure 6). The inset shows the amplitude of f 2 around 
R ~ 8 h' 1 Mpc, i.e. <rf , which is in good agreement with the 
values given above. 

Figure 8 compares the values of f 2 (-R) for the model 
with a\ = —4 with the ones for the model proposed by 
Maddox et al. 1990a, with a\ = —2. The net effect is a 
higher amplitude for the latter model, with erf ~ 1.0 instead 
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Figure 8. Comparison of two different estimations of £ 2 from 
direct inversion of the angular variance ~2(0) using different se- 
lection functions (e = 0). Filled squares correspond to a\ = — 4, 
while open squares correspond to the model by Maddox et al. 
1990a, which has a\ = — 2. The inset is an enlarged view of a 
portion of the same graph around R ~ 8 h~ 1 Mpc. 



of erf ~ 0.8. In both cases </>* = 0, M* = -2, e = and 
Qi = 1. 

^From Figures 7-8 we can see that the amplitude of 
f 2 (-R) ) an d not the shape, is affected by the uncertainties 
in the selection function. 



5.2 Higher order correlations £j 

5.2.1 The amplitudes rj 

We next find the projection coefficients rj using equa- 
tion (7). In Figures 9 we show the values of r 3 as contours 
in the a\-M* plane for different values of e and (f>* = 0. We 
find similar results for other values of (f>*. 

The variation of r 3 for different models of the selection 
function, (i.e. in the a\-M* plane) is quite small, typically 
Ar 3 ~ 0.01 or smaller within the allowed regions. This vari- 
ation is smaller than that involving the 2-point function am- 
plitudes, e.g. ro. This is because T3 corresponds to a relative 
amplitude, i.e. {3/^2 > an d therefore there is some degree of 
cancelation between the values of Ik (note that rj are di- 
mensionless in powers of Ik while ro is not). 

The variation for different clustering models, i.e. as a 
function of e, is more significant. For e = we have T3 ~ 1.19 
(r 3 ~ 1.20) for Q = 1 (fio = 0), while e = -1.3 gives r 3 ~ 
1.169 (r 3 ~ 1.176) and for e = 1.3, r 3 ~ 1.217 (r 3 ~ 1.226). 
Thus the overall range is quite large, r 3 ~ 1.17 — 1.23, but 
still only represents a 5% variation compared to the 32% 
variation in ro in Figure 6. 

The general pattern is similar for higher orders with the 
overall variation increasing to about 10%, 15%, 21%, 28%, 
34% and 41% for J = 4,5, 6, 7, 8 and 9, respectively. 




CO 



o 
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Figure 9. Contours of values of r 3 for the APM bj = 17 - 20 
galaxy sample using different luminosity functions, i.e. values of 
ai, M* , and </>* = 0. Each line marks the boundary for contours 
of T3 values, as marked. Each panel shows a different model for the 
evolution of clustering labeled by e and Q (Q = 1 except for the 
top right-hand corner, which has Q = 0.2). The contours show 
the allowed regions according to the number counts and mean 
redshift of Figure 3. 



5.3 The shape of Sj 

We next use the inversion presented in §2 to estimate Sj 
from sj at different scales. Figure 10 compares two extreme 
cases: i) Q = 1 and e = —1.3 (open squares) ii) Q = 
and e = +1.3 (filled squares). In both cases (f>* = and 
M* = —2. As pointed out above, the net effect of changing 
the selection function is small and the differences in Figure 
10 are dominated by the changes in e. 



6 DISCUSSION 
6.1 The value of <7 8 

Our final set of selection functions differ from that proposed 
by Maddox et al. (1990a) for inverting the APM 2-point 
correlation, and also used by Gaztanaga (1994) to estimate 
higher order correlations. The difference, though small, is 
significant given the errors. The net effect is a higher ampli- 
tude for the selection function proposed by Maddox et al. , 
with erf ~ 1.0 for e = 0, instead of erf ~ 0.8 (see Figure 7). 
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Figure 10. Comparison of two different estimates of Sj(R) from 
direct inversion of the angular variance sj(6) using different se- 
lection functions. Points show the two most extreme cases, as in 
Figure 7. Lines correspond to the perturbation theory predictions 
for the linear power spectrum P(K) of the T = 0.2 CDM model 
(solid line), and the estimated P(k) in the APM (Baugh & Efs- 
tathiou 1993) (dashed line). 



The effect on the hierarchical amplitudes Sj for the higher 
order correlations is smaller. On the other hand, our best 
models for the selection function (cf Figure 3) agree well 
with the functional form for the redshift distribution N(z) 
proposed by Baugh & Efstathiou (1993). However, the latter 
does not explore the uncertainties in the selection function. 

If galaxy clustering in the APM grows according to the 
gravitational growth of matter fluctuations (e ~ 1.3), then 
a s APM = 0.95 ±0.07 (1.00 ±0.08) for Q ~ 1 (fio ~ 0), close 
to the values measured from nearby optical samples such as 
the North Zwicky Center for Astrophysics catalog (Huchra 
et al. 1983, hereafter CfA) or the Southern Sky Redshift 
Survey (Da Costa et al. 1991, hereafter SSRS). On the other 
hand, if we assume that clustering evolution is fixed in co- 
moving coordinates (e ~ —1.3), we find a s AFM = 0.83 ± 0.05 
(0.87 ± 0.06), closer to the value for IRAS galaxies (e.g. 
Fisher et al. 1994). On comparing with redshift samples, 
one should take into account the possible effect of peculiar 
velocities. Fry & Gaztanaga (1994) have used configurations 
that minimize redshift distortions to estimate the clustering 
in real space in the CfA, SSRS and 1.9 ly IRAS catalogues. 
For our comparison we focus on their results for volume 
limited samples CfAN80, SSRS80 and IRAS65, which rep- 
resent a compromise between sampling a large volume and 
having a large enough galaxy density. The clustering in the 
smaller nearby samples is probably affected by large scale 
density fluctuations. We use the quoted mean values of ro 
and 7 and its uncertainties from Table 4 of Fry & Gaztanaga 
(1994) to find as from equation (4). 

The resulting values for as are shown in Table 2. These 
values are compatible with the estimates made by Fisher 
et al. (1994) for the 1.2 ly IRAS and those for the CfA 



Table 2. Values of as in real space estimated for CfA, SSRS and 
IRAS samples volume limited to a depth D, and covering a total 
volume Vt in ( /i -1 Mpc) 3 . 



Sample 



V 



V T 



as 



CfA 

SSRS 

IRAS 



80 h 
80 h' 
65 h 



- 1 Mpc 
- 1 Mpc 
- 1 Mpc 



3.0 x 10 5 
2.9 x 10 5 
1.0 x 10 6 



(from Davis & Peebles 1983) 



CfA 



IRAS 
&S 



1.21 ± 0.21 
0.97 ± 0.25 
0.70 ± 0.21 

0.69 ± 0.04 and 

0.95 ±0.06, but we find larger errors. This could be 
partially due to differences in the method of estimation. The 
variance coming from the finite size of the sample (i.e. caused 
by fluctuations on the scale of the sample) could be quite 
important for these nearby catalogues. Fry & Gaztanaga 
(1994) do include a finite volume contribution to the error 
by modelling the tail of the probability distribution. In fact, 
the difference between the estimates of as from the CfA and 
SSRS samples in Table 2 and also the difference between 
the two estimates mentioned for a^ A (which use different 
subsets of the CfA galaxies), indicate that the effect of the 
combination of finite volume and (possible) differential se- 
lection biases for optical galaxy samples is as large as 25%. 
This is in good agreement with the error estimates in Table 
2. 

The precise values of as Irom the APM galaxies provide 
important constraints on models for structure formation. For 
example, there is a whole range of models for which the pre- 
dicted values of as lor matter fluctuations in linear theory, 
a^ , turn out to be a\ > 1 when normalized to COBE fluctu- 
ations (Gorski, Stompor & Banday 1995, Stompor, Gorski & 
Banday 1995, Gorski et al. 1995). Given that the non-linear 
values of clustering are typically similar or larger than the 
linear values at 8 ft -1 Mpc (see Figure 11-12), we see that the 
amplitude of matter fluctuations in these models are larger 
than the fluctuations estimated from APM galaxies in Ta- 
ble 1. Thus these models require that galaxies are anti-biased 
with respect to the mass, i.e. that galaxy formation (and/or 
galaxy selection) is less likely to occur in large scale high 
density peaks and more likely to occur in large scale low 
density voids. Although not impossible, this would require 
a peculiar selection effect which needs to be studied. 

In Figures 11 and 12 we compare the APM variance 
f 2 (open squares) for the case with highest amplitude, 
i.e. e = 1.3, with the linear (continuous line) and non-linear 
(closed squares) predictions for matter fluctuations in the 
T = 0.2 (Qo = 0.2, A = 0.8, h = 1) CDM model and the 
T = 0.5 (Qo = 1, h = 0.5) CDM model (from Gaztanaga 
& Baugh 1995). Both models are normalized to a\ = 1, 
which corresponds to the lower end of the COBE normaliza- 
tion even when a small tensor contribution is allowed. The 

• j 2 —APM .—CDM 

inset shows the bias factor b = Q 2 /S2 as a iunc- 

—C DM ... .. , 

tion ol scale, where £ 2 is either the linear (continuous 
line) or non-linear matter variance (closed squares). For a 
local biasing transformation, one would expect that at large 
scales, where £ 2 < 1, there should be a linear relation be- 
tween galaxy and matter £ 2 , independent of scale (see Fry 
& Gaztanaga 1993). This does not seem to happen in the 
"standard" CDM model (Figure 12), which is the reason why 
this model is not favored by the clustering of APM galaxies 
(Maddox et al. 1990a). The open model (Figure 11) shows 
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Figure 11. Comparison of the APM galaxy variance £ 2 (open 

squares) with the variance in the matter fluctuations of the T = 

0.2 CDM model (fi = 0.2, h = 1, <r 8 = 1) measured from an N- 

body simulation (filled squares). The continuous line corresponds 

to the linear theory prediction for the same model. The projection 

effects in the APM have been modelled so as to give the highest 

amplitude. The inset shows the ratios of the galaxy variance to 

_ • ,9 -;APM .-CDM , . 

the matter variance o = £ 2 /S2 ( on a hnear scale). 

a constant b 2 for R > 10 ft -1 Mpc, but its value is smaller 
than one at all scales (anti-bias).^ 

6.2 Higher order correlations 

The final uncertainty in the range of values of Sj is typi- 
cally twice the estimated sampling errors, with the highest 
values for the case of less clustering evolution (Figure 10). 
Again, the precise values of Sj from the APM could provide 
important constraint on models for structure formation. If 
galaxy fluctuations trace matter fluctuations, one can pre- 
dict the values of Sj in the APM at large scales by just 
assuming gravitational growth from initially small gaussian 
fluctuations. These predictions depend on the shape (but 
not the amplitude) of the initial power spectrum, and are 
only valid at large scales £ 2 < 1 or R > 10 ft -1 Mpc (Fry 
1984b, Juszkiewicz, Bouchet & Colombi 1993, Bernardeau 
1994, Juszkiewicz et al. 1995, Lokas et al. 1995, Gaztanaga 
& Baugh 1995, Baugh, Gaztanaga & Efstathiou 1995). They 
are shown as continuous lines (APM final power-spectrum) 
and dashed lines (low Q CDM initial power-spectrum) in 
Figure 10 (from Gaztanaga & Frieman 1994). 

Bernardeau (1995) has critized the use of the "tree" 
hierarchy (2) to model the projection effects in angular cat- 
alogues because the PT results are not exact tree models 

t Although 6 2 is close to unity at large scales in Figure 11, the 
mean COBE normalization for this model without any tensor con- 
tribution is cr^ ~ 1.3, i.e. £ 2 should be scaled up a factor ~ 1.7 
producing b 2 ~ 0.5 




10- 3 ^ | 10 100 

1 10 100 

R (Mpc h- 1 ) 



Figure 12. Comparison of ^2( r ) vames in the APM survey and 
for the T = 0.5 CDM model (fi = 1, h = 0.5, <r 8 = !)• The format 
and symbols are the same as in Figure 11 . 

with constant values of Qj. Bernardeau favours using the 
2D projected PT predictions directly in comparisons with 
the angular data. The problem with this approach is that 
the PT results are only valid on large scales while the projec- 
tion mixes small with large scales. ^From the N-body results 
mentioned above it is clear that, at least at scales where 
£2 > lj the PT hierarchy has the wrong amplitudes. This 
could introduce spurious projection effects, which are par- 
ticularly bad for the PT hierarchy as it is not scale invariant. 
Moreover, the small angle approximation 9 — > has a dif- 
ferent meaning when the hierarchy is not scale invariant, 
which could explain why the Bernardeau results seem to be 
so sensitive to this approximation. In contrast, the quasi- 
scale invariant approach presented here (and used in G94) 
is optimal in the sense that it allows the hierarchy to be 
scale dependent (as in the PT case) but without introduc- 
ing the spurious amplitudes at small scales. Our preliminary 
tests of projections in N-body simulations seem to verify the 
validity of the quasi-scale invariant model. 



7 CONCLUSION 

We have studied the uncertainties involved in the estimation 
of three dimensional clustering properties from the angular 
distribution in a sample b j = 17 — 20 from the APM Galaxy 
Survey. We have considered the effect of a change in the 
selection function and the changing of clustering amplitudes 
with redshift. 

The final range of values for the inverted 2-point ampli- 
tude, covering all possibilities, is quite large, as ~ 0.78 — 1.08 
(see Table 1). The predicted shape for £ 2 is n °t much affected 
by any of these uncertainties (see Figures 7-8). The values 
of as in the nearby samples, Table 2, can be compared di- 
rectly with those in Table 1 for the APM Survey. We can see 
that the sampling and projection errors in as for the nearby 
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catalogues are two to three times larger than those in the 
APM, but the uncertainties in the cosmological parameters 
and the evolution of clustering are less important. 

In a recent preprint, Loveday et al. (1995) have esti- 
mated erf = 0.90 ± 0.05 for the real space variance in the 
Stromlo/APM Redshift Survey, i.e. a s = 0.95±0.03. A com- 
parison with the values in Table 1 suggests a large value of e, 
i.e. as expected if clustering grows according to gravity in an 
expanding universe with little or no biasing. In particular, 
the value of as in the Stromlo/APM is clearly incompatible 
with the one in Table 1 for the whole APM if e = —3, indi- 
cating that both the Universal expansion and some degree 
of clustering evolution are neccessary to reconcile the two 
measurements of as . 

The results for higher order correlations in Figure 10 
agree with the results in Figure 2 of Gaztanaga & Frieman 
1994, where it was pointed out that the observed values of 
Sj in the APM Survey are compatible with the clustering 
that emerges from gravitational growth of small (initially 
Gaussian) fluctuations, regardless of the cosmological model 
we assume for the universe, i.e. Q, A, Ho or the nature of dark 
matter. Again we find here that the observations requiere 
little or no biasing to match the gravitational predictions. 

Some of the uncertainties considered in our analysis 
could be removed with a better knowledge of the redshift dis- 
tribution N(z) for galaxies in the bj = 17 — 20 range, while 
a better understanding of galaxy formation seems necessary 
to make more detailed predictions about galaxy clustering 
evolution and the possibility of bias or anti-bias in the final 
galaxy distribution. 
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